Finite Sample Based Mutual Information

Mutual information is a popular metric in machine learning. In case of discrete target variable and continuous feature the mutual can be calculated as sum-integral weighted log likelihood ratio joint marginal density distributions. However, practice true distributions are unavailable only finite sample population given. this paper, we propose novel method for calculating variables using population. The proposed based on approximating underlying distribution Kernel Density Estimation. Unlike previous kernel-based approaches estimating information, our calculates directly integral involved formula. Numerical experiments demonstrate that produces more accurate results than currently used selection approaches. addition, demonstrates substantially faster computation times benchmark methods.